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Web-based version. Nevertheless, 21 of these 29 responses were received 
before there were any responses to the traditional survey. Most respondents 
approved of the idea of electronic surveying in spite of the technical 
difficulties. The other two pilot studies, one asking principals about block 
scheduling and one studying the ability of students to respond to an 
electronic survey, also showed favorable responses and very quick reply time. 
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Introduction 

Since 1996 and the pioneering efforts of a few market 
research concerns (Kottler, 1998), interest in the use of the 
Internet as a vehicle for survey research has increased 
dramatically. For observers like Dillman (1998) , such interest is 
a concomitant of a general methodological trend away from 
interviewer-controlled and towards self-administered surveys of 
all types. Then too, the size of the Internet community is growing 
and its composition becoming more representative of the general 
populace. According to a recent report appearing in USA Today 
(Sefton, 2000), the number of this nation's Internet users has 
shot up to 106 million or slightly more than 52% of the total 
population of American adults. While men remain a slight majority 
and Black Americans are still underrepresented, the Angus Reid 
Group projects not only that the Internet gender gap will 
disappear by the end of the year 2000 but also that "the number of 
new users from the baby boomer generation will surpass new teen 
and young adult users for the first time" and that " [h] ouseholds 
with children will also log on in greater numbers than ever 
before" (Nua Ltd., 2000) . Clearly, the time when the Internet may 
be used as a medium for querying more than special populations 
with unusual talents and unique interests -- heretofore the 
typical targets of web- and email-based research (see, for 
example, Farmer, 1998) -- is drawing nearer. 

Moreover, heightened enthusiasm for online surveys has also 
been sparked by an increasing number of firms offering technical 
services, off-the-shelf software packages, or -- as the case of 
SPSS/MR -- both, in order to make what once was an involved and 
daunting task one much less opaque and easier to achieve. Because 
typical web pages are simply "served up" to the viewer, the 
creator of such pages has only to know enough HTML (Hypertext 
Markup Language) to construct a readable and aesthetically 
pleasing facade. However, because so-called Internet or web 
"forms" are genuinely interactive -- meaning that they both accept 
and process data input from the user -- an additional layer of 
complexity is involved by way of some back-end programming. To 
work, such programs or scripts (i.e., executable computer code 
commonly known as "cgi-bin" files) call not only for more than 
just a passing acquaintance with a specific scripting language -- 
ordinarily but not necessarily the UNIX-compatible PERL -- but 
also for freer access to and increased demands upon the so-called 
web "server." 
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Given these two sets of obstacles, use of the Internet as a 
data collection tool seemed out of reach for all the most 
accomplished and outfitted researcher until a number of firms 
began developing solutions to what were essentially problems of 
technical expertise on the one hand and problems of control on the 
other. Providing software that automates the creation of back-end 
executable code and/or technical consulting for custom tailored 
solutions, many firms have now rendered the programming issue 
largely moot, opening the door to many more web surveys, more 
efficiently executed. Control-related issues have been similarly 
dispatched, with many firms offering clients access to a remote 
server where the survey may be hosted and the results processed 
and stored. While the level of costs incurred are typically 
commensurate with the additional sorts of services involved, even 
firms producing off-the-shelf survey solutions include data 
processing and a modicum of storage on their own servers in the 
cost of the software. Under this arrangement, individuals having 
only an email account and the usual user privileges afforded by 
commercials ISPs (i.e., Internet Service Providers) can be up and 
hosting web and email surveys 

Yet, in the final analysis, it is largely because of a host 
of presumed advantages that has led many an interested researcher 
to consider the use of the Internet as a viable alternative to 
more traditional modes of survey administration. A tireless 
promoter of what he refers to "an unprecedented tool for data 
collection," one such researcher -- W. C. Schmidt (1997a)-- has 
not only offered a number of position papers and technical briefs 
to the social science community on electronic surveying, he has 
also made available to colleges and university a free-of-charge, 
Java-based utility — WWW Survey Assistant — that simplifies and 
customizes the creation of web-based forms and accompanying cgi- 
scripts (1997b) . If straying at times into hyperbole, his comments 
are indicative of the many claims advanced To begin with, notes 
Schmidt (1997a), 

The costs in terms of both time and money . . . are 
low compared with the costs associated with conventional 
surveying methods. The data entry stage is eliminated 
for the survey administrator, and software can ensure 
that the data acquired from participants is free from 
common entry errors . Importantly, web surveys can 
interactively provide participants with customized 
feedback . These features come at the price of ensuring 
that appropriately written software manages the data 
collection process. Although the potential for missing 
data, unacceptable responses , duplicate submissions, 
and web abuse exist, measures can be taken when creating 
the survey software to minimize the frequency and 
negative consequences of such instances, (p. 274) 

To determine the authenticity of claims like Schmidt's, a 
project was designed to compare traditional paper-and-pencil 
methods with several electronic alternatives. When several of 
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these alternatives were ruled out either because of known 
incompatibilities with the organization's email system, variation 
in the kinds and configurations of computer workstations across 
the organization, or variation in the computer literacy of the 
group being surveyed, the competing methods were limited to only 
two: the paper-and-pencil "control" and the web-based 
"experimental." The purpose of the bulk of this paper is to 
outline the results of this test. In an addendum, two other pilot 
studies subsequent to the first are discussed, these also 
employing electronic methods, but with no or only minor 
comparative intent. 



Review of Relevant Literature 

The rapid rate of change in Internet technology coupled with 
the equally rapid and ongoing evolution in the membership the 
online community tend to make most studies obsolete by the time 
they reach publication. However, there is a small body of 
published literature that is concerned with utility of email 
surveys and that is mostly characterized by equivocal findings and 
equivocal conclusions. Two 1995 studies by Metha and Sividas and 
Tse et al. independently compared email and postal surveys in 
different settings. Where the former found that email surveys were 
marked by quicker return, better quality of response, and a rate 
of return to comparable to postal surveys, the latter conducted in 
the Chinese University of Hong Kong observed a significantly lower 
rate of return with email (6%) compared with mail (27%) . 

Undertaken as part of project for the Henley Center for Media 
Futures, a subsequent study by Comley and his associates (1996) 
effectively split the difference in the preceding studies' 
outcomes. On the one hand, Comley observed a "major time advantage 
for email over postal methods" with his targeted population of 
readers of an Internet magazine. On the other, he received a 
somewhat lower response rate by email (13.5%) than by post 
(15 . 4%) , despite the fact the email group were sent reminders. 
Without it, Comley offers, the response rate would have been a 
mere 12.6%. 

While more recent studies do not dispute the potential 
savings in time achieved by electronic mail survey, they do take a 
different view of the assumed savings in cost. In a pilot study 
conducted on behalf of the Census Bureau, Nichols and Sedivi 
(1998) attempted to obtain economic data from a number of 
companies using a Computerized Self-Administered Questionnaire 
(CSAQ) which resided on a government server. In a manner similar 
to a Survey- to Disk technology, respondents given the URL (Uniform 
Resource Locator ) of the CSAQ which they then accessed over the 
web with the correct username and password. Unfortunately, the 
authors soon discovered that the complexity of the coding (a mix 
of HTML and Javascript) greatly limited the number of respondents 
who could simply access the CSAQ. Still worse, in cases where the 
respondents were more fully and correctly equipped to work with 
the CSAQ, the help desk was deluged with calls that often 



referenced browser configuration problems but more typically 
concerned mistyped URLs and lost usernames and passwords. As the 
number of CSAQ reporters who phoned in with difficulties topped 
56%, Nichols and Sedivi were led to question whether the method -- 
or at the least the software -- was viable "as the volume of calls 
could not be handled in a production setting" (p. 3) : if nothing 
else, the man-hours involved would render the per unit cost of the 
survey prohibitively expensive. Based on their experiences, 
Nichols and Sedivi concluded their study with a call for more 
research as both "[t]he response rate and the help desk findings 
suggest that there are different issues of motivation and 
assistance not present in paper questionnaires" (p. 7) . While 
their initial screening of respondents suggested that there issues 
of coverage issues were also salient, still Nichols and Sedivi 
"found enough positive respondent reaction to continue to pursue 
collecting data using the Web (p. 7) . 

Similar technical difficulties dogged and concomitant "help 
desk" overload attended an email/mail comparison conducted by 
Couper, Blair, and Triplett (1998) . Unlike earlier modal studies, 
this particular comparison was not confined to a single 
organization but rather spanned several statistical agencies of 
the Federal Government. While employees within each agencies were 
known to have email access, the peculiarities of each of the 
agencies' mail systems -- which ranged from Lotus CC : Mail to 
Novell Groupwise to WPMail and which differed, in turn, from the 
author's own system (Pegasus Mail) -- altered transmission just 
enough significantly to drive down response rate and markedly to 
drive up calls for assistance. In all, the authors noted that they 
"handled over 900 incoming toll free calls regarding the survey, 
most of them ... technical questions about email" (p. 9) and 
concluded that when added to "large start-up costs , " the per piece 
cost of the email survey was more expensive than regular mail 
given the response rate. While maintaining that "[e]mail clearly 
offers a lot of promise," Couper, Blair, and Triplett finally 
advise that, in their experience, "technical difficulties need to 
be overcome before [it] can be used routinely for surveys of large 
and diverse populations across multiple organizations" (p. 9) . 



Description of the present study 

Although the present study is more focused on the web as a 
medium for survey research than email, it nevertheless resembles 
the Couper et al . investigation in several respects. First, it too 
was embedded in a larger study that was concerned with 
organizational climate. Second, front-end changes had to be made 
in the design of the study when it was discovered that the 
organizational infrastructure was uncongenial to its conduct. In 
the Couper et al. study, their original intent was to use email 
messages that were HTML-based and that functioned much like an 
Internet form. At the eleventh hour, however, the authors were 
forced to forgo the Graphical User Interface (GUI) of HTML-based 
messaging in favor of a more awkward and unattractive ASCI I -based 
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solution that, for all its flaws, was still the lowest common 
denominator . 

As stated above, the study's initial plan was to compare 
conventional paper-and-pencil methods with several others produced 
by an inexpensive, off-the-shelf software package called Survey 
Solutions for the Web. As the name of the product suggests, the 
primary purpose of the software is to generate the requisite HTML 
and cgi-code for online surveys that are both single page (as 
opposed to those consisting of multiple screens) and noncomplex 
(i.e., without elaborate branching or "piping" based on user 
responses) . At the same time, the product can render the same 
questionnaire in several other formats that are email-based. 

These formats include both the aforementioned ASCII- and HTML- 
based messaging types, as well as CSAQs that can be administered 
on disk or attached to an email message and subsequently run on 
the user's own computer (run-time email). 

While run-time email was quickly eliminated as being too 
complex for most of the intended respondents to handle, high hopes 
were maintained for the other methods until pilot testing 
indicated otherwise. As in the Couper et al study, problems with 
the email system -- Lotus Notes Mail -- immediately ruled out both 
email forms. When ASCII-based email was attempted, Lotus Notes 
transformed the file into a continuous stream of text that was 
unreadable: a peculiarity of Lotus that was confirmed when the 
software vendor itself attempted to survey this writer. Likewise, 
HTML-based email was eliminated after various tries with 
Netscape's "send page" command. Expectedly, use of this option 
placed the questionnaire in the body of the email message in a 
format resembling a online form. Unexpectedly, however, recipients 
of the email message were treated to a questionnaire with the 
submit button gone and all response controls -- radio buttons and 
text buttons -- omitted. 

With the preceding choices eliminated, the only viable 
methods that could be compared included the diskette CSAQ and the 
web-based questionnaire. Because the diskette CSAQ required some 
release of Windows (either '95, '98, or 2000), it was shelved for 

later testing in environments where this particular platform was 
the de-facto standard. Although the decision was made to proceed 
with the web questionnaire -- the only option remaining -- here 
too problems surfaced. To control access to the survey, 
respondents were asked to click on URL link in the body of an 
email message. So doing would then activate the respondent's web 
browser — typically either release 3.x or 4.x of Netscape -- 
which would then open upon an introductory page. Once there, a 
flag or "cookie" would be written to the respondent's hard drive 
to discourage multiple submissions during the survey period and 
then the survey would itself appear in a second window after 10 
seconds . 

Written in the simplest dialect of Javascript, these added 
security features seemed to work flawlessly in pilot testing with 
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all browser versions -- both Netscape and Internet Explorer -- 
numbered 3.0 or above. What pilot testing also revealed, however, 
was that, on a significant number of workstations and seemingly at 
random, Lotus Notes had been improperly configured. Rather than 
Netscape -- the organization's standard — Lotus was instead set 
up to employ some other browser either unimplemented by the system 
(Lotus's own web navigator) or present on the respondent's hard 
drive (Internet Explorer) but not configured for Internet access. 

After further investigation proved that this was indeed the 
primary source of the difficulty, it was nevertheless decided to 
proceed with the study despite the additional level of complexity 
introduced and inconvenience to the respondent entailed. For those 
respondents directed to the web, instructions had to be rewritten 
(and much lengthened) both to alert them to the problem and to 
offer them another, insecure mode of access that involved evoking 
Netscape manually. In addition, the instructions also included an 
offer to help the respondent "fix" Lotus Notes, if desired, by 
contacting the author by phone. At the onset of the study, none of 
these instructions were, however, written down or distributed to 
either the "control" or the experimental respondent groups. — 



Sampling Procedures 

With "customer satisfaction" being the substance of the 
study, all district principals of those schools having regular 
contact with the central office unit in question were defined as 
the target population. Subsequent to this in-house screening 
procedure, 160 principals were selected and randomly assigned to 
one of the two conditions, using a combination of the SAS 
"uniform" function -- which assigns random numbers -- and the SAS 
procedure "PROC RANK" -- which sorts the respondents into a 
specified number of groups based on the rank order of the random 
numbers. After a general memo describing the study's content and 
form was released, two members each of both groups were asked not 
to be considered given their interim administrative status and 
their inability to respond to survey fully. Thus, with these four 
potential respondents screened (there may, in fact, have been 
others who did not take the trouble to contact the office) , both 
"control" and "experimental" groups were of equal size, consisting 
of 78 members each. 



Survey Design and Structure 



Concerned with "the quality of services rendered to schools 
by the Office of Research and Evaluation, " both paper and 
electronic forms of the questionnaire were kept brief. Each was 
divided into three major sections having their own instructions. 

In Part I, responding principals were asked to rate eleven unit 
behaviors on a continuous scale, according to each behavior's 
perceived frequency of occurrence. Both the width of the scale and 
the content of the first four items were expressly designed to 
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recall "the Competing Values Organizational Effectiveness 
Instrument" (Quinn, 1988) that had been previously administered to 
all unit members as a vehicle for intra-departmental discussion 
and self-study. In their turn, these four "between unit" items 
were supplemented with seven other "within-unit" ones that emerged 
from a tally of unit member's reactions to a series of customer 
satisfaction prompts. In an open forum, unit members -- whether 
clerical or administrative -- elaborated on their choices, 
suggesting specifics to be used in item development. When a set of 
seven core issues were agreed-upon, the author crafted the items 
and resubmitted these to unit members for any corrections and 
additions . 

While the items constituting Part I of the survey were 
concerned with customer needs, those making up Parts II and III 
were directed at customer wants and preferences. Arrived at in the 
forum described above, the four Part II items were phrased as 
open-ended questions and sought information about improving 
specific unit products — its workshops, written materials, and 
website -- and general commentary on the unit. Concluding the 
survey, the items constituting Part III were concerned with an 
office "stretch goal": an unlooked-for, unrequested service that 
the unit could provide that might "surprise and delight customers" 
(Cameron & Quinn, 1999). Underwritten by a "continuous 
improvement" or "kaizen" approach to service quality, these last 
few items were somewhat obliquely stated but designed to 
complement the TQM perspectives addressed in Parts I and II where 
manifest customer needs and felt customer preferences were, 
respectively, expressed. 

In both surveys, a final, single item was devoted to the 
modal aspect of the study. Regardless of their group status, all 
respondents were asked to rate the level of convenience of 
completing a survey online. While the content of both forms of the 
item was the same, the phrasing differed such that the 
"experimental" group was asked about an experience that they just 
had, while the "control" group was probed about an experience they 
might have in the future. 



Results of the Modal Study 

Similar to findings in previous studies, most respondents 
seemed to approve of the idea of electronic surveying despite 
technical difficulties. Indeed, one principal who had to be 
contacted by phone because of a failure of the infrastructure — 
that is, non-delivery caused by the system hardware, not the Lotus 
Notes software -- told this writer than if principals had to "keep 
on doing these surveys, then this was the way to do them." In 
terms of the actual percentages and discounting two missing 
"control" group responses, exactly 63.5% (n =47) believed the 
process of online surveying to be either "much more convenient" or 
"more convenient" than the conventional method, some 20.3% (n = 

15) were indifferent as to mode, and only 16.4% (n = 12) found 
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enough flaws in the new modality to rate it as "less convenient" 
or "much less convenient" than paper -and-penci 1 . Interestingly, on 
this particular directed response item as well as the eleven 
others, no effect by delivery mode was observed. 

Nevertheless, the Lotus Notes issue outlined above seemed to 
have the expected devastating impact on response rate. While the 
combined returns were poor -- a mere 76 out of a possible 156 or 
only 48.7% -- those garnered via the web were especially so. Where 
almost 63% of the "control" group responded to the questionnaire 
without follow-up, only some 37% did so online. While these 
findings are coincident with those of others previously described, 
on a more positive note, so were speed of response findings. 

Before a single posted response had been received from the 
"control" group, some 21 of the 29 ultimately obtained from the 
"experimental" group -- or roughly 72% -- were already in hand. 
When written directions on configuring Lotus Notes were ultimately 
circulated, the remaining online responses received were obtained 
within the space of two days. 



Conclusion 

In historical perspective, it took all of four centuries for 
textual literacy to attain near universality and several decades 
for the telephone. to become a taken- for-granted household fixture. 
When looked at in this way, it is hardly surprising, then, that 
email- and web-based methods of surveying individuals might take 
some time to. mature. 

As this and earlier studies suggest, an array of technical 
difficulties presently complicate valid studies conducted online, 
resulting in higher error rates in coverage and in nonresponse. 
Until the technology matures, it is recommended that researchers 
invest in multi-modal approaches, mixing methods like paper, 
email, and web surveying with little consideration for differences 
between venues. While responding to an online survey may indeed 
entail what Di liman (1998) refers to as "computer logic" as well 
as "questionnaire logic," this may be a difference bound for one 
without distinction as the online community grows, as computer 
literacy increases, and as communication across the Internet 
becomes a universal fact of everyday life. 



Addenda: Pilot Studies Two and Three 



The qualified success of the preceding study prompted further 
examination of electronic surveying by other means, with different 
populations, or both. In a second modal study that involved three 
different forms of block scheduling currently operating in the 
Memphis City Schools, a version of the CSAQ output by theSurvey 
Solutions for the Web software was used in questioning faculty 
about the advantages and disadvantages of the block under one of 
three different plans: the Alternate Day Plan, 10 Day Rotation 
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Plan, and the "semestering" or 4 X 4 Plan. Given the prevalance of 
the "Windows machine" at all of the district's secondary schools 
and the fact that the generated CSAQ was specific to the Windows 
platform, the decision was made to proceed with this second modal 
study, pending principals' declaration of ready access to 
computers by faculty. After this had been established, a master 
copy of the survey- to-disk was used to mass produce approximately 
500 copies -- this number including a "fudge factor" in the 
instance of bad copies or underestimates of faculty size -- and 
the individual copies placed in individual mailers. Also 
containing a page of written instructions, these individual 
mailers were then boxed up and mailed by schools to schools at the 
end of March 2000. Because of the intervening "spring break," 
principals and faculty members were told that all returns were 
expected by the end of April 2000. To simplify data collection, 
principals were asked to appoint a survey administrator to oversee 
the initial distribution, ongoing collection, and subsequent 
return of all instruments completed at the school together. 
Moreover, it was requested that unused materials be separated from 
used ones and returned in a second box designated specifically for 
that purpose. 

Based on estimated faculty size, the response rate of usable 
CSAQs was observed to vary markedly by school, with three of the 
nine schools returning little more than 50% of the instruments 
completed, but wi-th others with a completion rate in excess of 
70%. In the aggregate, this "observed response rate" was roughly 
76% of the "estimated response rate" (N = 342). In counting simply 
unused and uncompleted instruments, it was further estimated than 
in excess of 90% of the materials sent -- disks and mailers -- had 
been returned in re-usable condition. 

As to respondent reactions, building-level principals and 
survey admininstrators indicated that faculty, generally speaking, 
"had no trouble" with the format. Indeed, some even indicated 
their preference for it, mentioning the attractiveness of the 
user-interface, the additional level of confidentiality afforded, 
and/or the additional space for open-ended commentary allowed. Of 
more than 700 such open-ened comments obtained, only one -- a 
negative one -- made reference to the alternative mode of 
presentation. For this individual, the type was too small, perhaps 
a function of his/her screen resolution. 

Returning to the web-based form, a final modal study that was 
conducted at the end of the academic year 1999-2000 investigated 
the ability of elementary school children to obtain access to and 
interact with an electronic questionnaire. Both the regional 
accreditation process and "data-driven school improvement" notion 
recommend that student perceptual data be gathered, the latter on 
a regular basis. To determine whether such data might be more 
easily collected used electronic means, a "canned" survey was 
converted into html, decorated with attractive graphics, and 
uploaded to the web. After one K-6 school consented to participate 
in the pilot, a sample of students were systematically selected 
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from each class in grades 3 through 6, the total representing 
about 20% of the entire student body. Class by class, five or six 
children were escorted to the library at a time and then seated at 
a computer where the questionnaire's introductory page was up and 
running. With some help from the researcher and -- in one 
instance, some on-the-spot assistance from the school's ESL 
teacher -- all sampled students were able to complete the 
questionnaire successfully. Afterwards, students addressed a two- 
item questionnaire asking how much they had liked the electronic 
format compared to the usual way of answering on paper. No doubt 
due in significant measure to the "novelty" effect, the results 
were 100% in favor of the electronic form Indeed, not a few 
students suggested that they do all of their "tests" like that, 
"because its ' funer . ' " 

What was most indicative of the method's usefulness in the 
"data-driven school improvement" process, however, was the 
principal's reaction to the survey's "turn-around time." The 
survey finished early on a Friday afternoon, the data were quickly 
downloaded into the Survey Solutions program for initial 
processing and then into SPSS for the analysis. A printout was 
available for the principal's inspection the following Monday 
morning, and the results were used two days later as part of the 
discussion at an end-of-year faculty meeting. 
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APPENDIX: 

EXHIBITS OF MATERIALS USED 



MCS UNIT PERFORMANCE 
ASSESSMENT TOOL 

(based on the competing values framework) 

Listed below are sixteen statements that describe organizational performance. You should indicate 
how often they occur in your unit. Please use the following scale to respond to each statement. 

Place a number from 1 (meaning very infrequently) to 7 (meaning very frequently) in the space just 
before each of the items. Return this form to the the Office of Research & Evaluation by XX/XX/XX. 



Unit code number goes here. 

Return address is printed on 
the back of the form. 

v J 



Verynfrequently 1 2 3 4 5 6 7 — ► VeryFrequently 

— 1. The work process is coordinated and under control. 

_ — 2. Employees influence unit agenda-setting and decision-making. 

— 3. Rules, procedures, and formal methods guide the work. 

4. The goals are clearly understood by most members. 

— 5. The work effort may be described as intense. 

6. There is a stable, predictable work environment. 

7. Innovation is emphasized. 

8. There is a positive interpersonal climate. 

— 9. Quantification and measurement are key parts of the work climate. 
10. Consensual decision-making is encouraged. 

11. Outsiders perceive it as a vibrant, high potential unit. 

12. Creative insights, hunches, and innovative ideas are encouraged. 

— 13. It is easy to give an explanation of the overall objectives of the unit. 
14. Increasing productivity is a unit priority. 

15. Employees feel as though they really belong to the unit. 

16. The unit has the image of a dynamic, growing system. 



I. 

II. 




According to MCS salary schedule criteria, my position may best be described as: 
A. Administrative B. Clerical C. Other 

(pleases pec ify) 

My position involves managing the work of others: 

A. YES.it does. 
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B. NO, it does not. 



CUSTOMER SERVICE AUDIT 2000: 

SERVICE DELIVERY PROMPTS 



Using the prompts listed below -- not all of which are applicable to our office -circle the top 
six that you consider the most important with regard to the products/services that we render 
or might render to schools. The prompts you select could reference some 

(1) basic customer requirement or need addressed by our office; 

(2) some unmet but known customer preference or want that we ought to address; or 

(3) some existing product or service presently offered by us that could be enhanced in 

some small way. 

In light of these perceived (1) needs, (2) preferences and (3) potential ehancements, use 
the prompts you selected to suggest three possible items we could use in developing a 
short customer satisfaction instrument. On the back of this sheet, write these in the blanks 

provided. 



accessible 

accurate 

after-delivery 

service 

alternatives 

attentive 

attractive 

authentic 

available 

available inventory 

careful 

caring 

cheerful 

clean 

comfortable 
committed 
competitive 
concerned 
consistent 
convenient 
cost-effective 
courteous 
creative 
customized 
dedicated 
dependable 
discreet 
easy to do 
business with 
easy to locate 
easy to use 
effective 



efficient 

elegant 

empathic 

entertaining 

exciting 

expensive 

experienced 

experts 

extra amenities 

fair 

fast 

flexible 

friendly 

healthy 

helpful 

homey 

honest 

industry leaders 
influential 
innovative 
interesting 
intimate 
knowledgeable 
large in size 
leading-edge 
technology 
listens 
low cost 
medium in size 
moderate in cost 
modern 
on time 



personal 
pleasant 
pleasurable 
private 
professional 
quick 
quiet 
relaxing 
reliable 
respectful 
responsible 
responsive 
reputation for 
excellence 
safe 
serious 
sincere 
skilled 
small in size 
special 
stimulating 
technical support 
timeliness 
tranquil 
trustworthy 
understanding 
unique 
up-scale 
warm 

well-known 

well-planned 

well-stocked 
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CUSTOMER SERVICE AUDIT 2000 

Quality of Service Delivery Worksheet for 

Your Name: 

Name of Unit: 



What areas of customer need are addressed by our office? 

Here the emphasis is on meeting basic customer needs and requirements in order to reduce customer 

dissatisfaction and minimize complaints. 



AREAS I: 



Regarding our products/services, what are some customer wants ? 

Here the emphasis is less on the standard requirements of customers than on their preferences and 
expectations. A more proactive way of thinking about service quality, the goal is not to find and fix errors 

but to head them off. 



AREAS II: 



Quality Concern III: What product/service enhancements could our office 
provide that would surprise and delight our customers? 



Typical of some of the best firms in the 1990s, the emphasis here is on coupling continuous improvement 
— sometimes referred to as “kaizen” — with innovation so that current standards of performance are in a 

state of constant evolution. 



AREAS III: 



SERVICE DELIVERY PROMPTS 
TALLIES OF RESPONSES 



S . D . PROMPT 


DC 


LU 


LF 


BG 


DG 


VG 


LJ 


GK 


LM 


JN 


BW 


TOT 


ACCURATE 






1 


1 


1 


1 


1 




1 


1 


1 


8 


TIMELINESS 




1 




1 




1 


1 


1 


1 


1 




7 


RESPONSIVE 




1 


1 


1 








1 




1 




5 


HELPFUL 


1 




1 


1 




1 












4 


PROFESSIONAL 










1 






1 


1 


1 




4 


TECH SUPPORT 






1 


1 








1 






1 


4 


EFFICIENT 










1 






1 


1 






3 


KNOWLEDGABLE 






1 


1 




1 












3 








S . D . PROMPT 


DC 


LU 


LF 


BG 


DG 


VG 


LJ 


GK 


LM 


JN 


BW 


TOT 


COURTEOUS 




1 










1 










2 


CUSTOMIZED 
















1 


1 






2 


DEPENDABLE 




1 










1 










2 


EASY TO DO BUSINESS W/ 




1 










1 










2 


FAST 




1 










1 










2 


FLEXIBLE 










1 












1 


2 


RELIABLE 










1 


1 












2 


LISTENS 


1 




1 


















2 


RE PUT AT I ON / E XC E LL ENC E 


1 




















1 


2 



S . D . PROMPT 



DC 



LU LF BG 



DG 



ACCESSIBLE 
AUTHENTIC 
CLEAR* (added) 
CONSISTENT 
EASY TO USE 
EFFECTIVE 
EXPERTS 
INNOVATIVE 
MODERATE IN COST 
ON TIME 
UNDERSTANDING 



VG LJ GK LM JN 



BW 



TOT 



1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 



19 

o 

ERIC 



CUSTOMER SATISFACTION QUESTIONNAIRE 

OFFICE OF RESEARCH AND EVALUATION 

Divided into three parts, the following questionnaire concerns the the quality of services rendered to schools by 
the Office of Research and Evaluation. In Part I you are asked to rate the frequency with which office behaviors 
or products evidence a concern for quality. Using the seven-point scale below — where 1 means very infrequently 
and 7 means very frequently — mark a number in the blank next to each of the 11 statements. 



Ver^nfrequently 1 2 3 4 5 6 7 — ► VeryFrequently 



1. Facilitating the educational mission of the district and its schools is a 
clear office priority. 

2. Routine transactions with the office are marked by miminal red tape and 
maximum efficiency. 

3. In my dealings with them, office staff members appear courteous, 
competent, and concerned. 

4. The search for ways to improve its products and services is characteristic 
of this office. 

5. When I have an immediate need for information, my request is promptly 
handled. 

6. When I telephone, satisfactory answers to my questions are provided 
either by the secretary or the first administrative staff member on the line. 

7. Office memoranda, directions, and otherwritten documents directed at 
schools are clear, consise, and readily understandable. 

8. Office reports about district conditions or various projects and initiatives 
within the district contain useful information for school-level planning and 
decision making. 

9. Office policies and procedures for completing and returning tests, 
questionnaires, and other sensitive materials are easy to comply with. 

10. Office-sponsored workshops enhance my school's capacity for data- 
driven improvement of our instructional program. 

1 1 . Technical support in doing, accessing, and understanding educational 
research and evaluation is available on demand. 
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PART II. Write a brief response to the following open-ended questions. 

1 2. The development of what sorts of research-oriented or data analtyic skills should 
drive future workshops? 



1 3. What additional data sources ought to be included in the MCS “School Profiles” 
to increase its usefulness? 



14. To reduce its present length and complexity, what, if any, information appearing 
in the MCS “School Profiles” ought to be eliminated? 



15. What are some topics and/or resources that should be available online though 
the departmental website? 



16. Other comments? (use additional sheets as necessary) 



PART III: Indicate your preference with respect to potential Office “stretch goals.” 

17. Would be it be more or less convenient for you to complete a survey like this 
online? 

More! Less! Neither! 

18. The office is considering the use of spreadsheet templates to help automate the 
computational aspects of the school improvement planning process: 

What is your computer platform? PC Macintosh 

What spreadsheet program do you normally use? 

Program Version Number 
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GENERAL INSTRUCTIONS: SURVEY-TO-DISK 



PRELIMINARY — To begin the SURVEY-TO-DISK process, start the PC and allow Microsoft 
Windows to load completely. Once Windows is fully in the computer’s active memory, THEN 
insert the Survey-to-Disk in the 3 1/2 inch floppy disk drive. Because this disk drive is the one 
that PCs always search first, it is conventionally known as the “A:” drive. 





r; 


zJtJ 


u.ifti, • 


Jl' -•£§ i 


rr * 

5 • j 

Ul 1 



S 


** it ! 


|§ 


< * 
1^1 


iO 


Ltd 

0*1 Us 


M} 

T«f k: 




If:! 





fsjwLia'"" ' 



(1) Once the floppy disk is seated in disk 
drive “A:”, look for the Windows desktop 
icon called "MY COMPUTER.” This icon 
is usually located in the topmost position, 
in the upper-left hand corner of the screen 

Once found, double-click on the “MY 
COMPUTER” icon to open up a window 
like that at the immediate left. Through 
this portal, the user can gain access to 
resources important to the entire com- 
puter system. 



(2) One potential resource depicted in 
the “MY COMPUTER” window is the 
contents of the disk placed in the 3 1/2 
disk drive. Double-clicking on that 
drive’s icon will reveal the contents of 
the inserted disk in a new window. 

(3) On an unused Survey-to-Disk, three 
.icons will appear in this new window — 
the critical one being the one labeled 
SURVEY. After double-clicking on 
SURVEY’S clipboard icon, the disk drive 
will whirr... 
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(4) ... and you’ll see the opening screen of the 
faculty survey. The text provides general 
directions for responding to the survey’s sixty- 
five items. The buttons below the text are for 
navigational purposes. Throughout the 
survey, the NEXT button will move you 
forward one screen. The BACK button will 
return you to the preceding screen. Use of the 
BACK button allows you to review and/or 
change an earlier answer. 



(5) Although you may quit the survey at any 
time by pressing the CANCEL button, no 
responses will be recorded until you reach the 
last screen. Here the FINISH button finally 
becomes active, its power signaled by the 
lightening bolt. Pressing the FINISH button 
writes your preceding responses to a 
document on disk and takes you out of the 
completed survey in an orderly fashion. 





Questions about the survey to disk may be directed to Mr. Lou Franceschini in the Office of Research 
and Evaluation by phone (325-5450) or by email (franceschinil@memphis-schools.kl2.tn.us) 
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